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Abstract 

Thisartidereportson thefindingsofa Delphi study conducted to dd:ermineres&rch needs in 
educational technology over the next five years The Delphi pand consied of 30 educational 
technology expertsfrom throughout theU nited Stateswhopartidpated in a three-round consn- 
susbuildingprocessvia theinternet. T he results of thiserewrch study providea framework of 
eight res&rch priority areasand ^edficresarch topicsfor thoxengagd in educational technol- 
ogy. (Keywords rewrch priorities learning, teachers modds strategies) 

INTRODUCTION 

Since the introduction of the computer into education in the 1970s, research- 
ers have investigated its effects on students, teachers, and learning environ- 
ments. Early studies were focused on demonstrating the impact of a particular 
technology or software on student achievement and student motivation. With 
increased access to computers and the Internet, researchers extended their ef- 
forts to investigate the role of technology in the educational setting, including 
its impact on teachers and the learning process. In the past decade, the study of 
the computer as an instructional delivery medium has been expanded to investi- 
gating technology as “a transformational tool and an integral part of the learn- 
ing environment” (Pouts, 2000, p. 9). 

A review of the literature discloses an abundance of educational technology 
research studies employing a variety of research methodologies in a variety of 
educational settings. In an effort to synthesize and analyze the results of these 
studies, a number of meta-analyses have been conducted. One of the more 
comprehensive meta-analytic studies (Kulik, 1994) summarized more than 97 
of the computer-based instruction studies conducted in the 1980s, noting that 
students typically learn more and faster in courses involving computer-based in- 
struction and have more positive attitudes. The results of his meta-analysis sup- 
port the use of computers as a means to improve student achievement. 

Schater (2001) analyzed a number of large-scale studies of educational tech- 
nology and concluded that students in technology-rich environments experi- 
enced positive effects on achievement and a consistent improvement in self- 
concept. Moreover, his research supports the use of computer assisted 
instruction and collaborative networked technologies as a means to teach 
higher-order thinking. 

Waxman, Connell, and Gray (2002), in a report commissioned by the U.S. 
Department of Education, presented a quantitative synthesis of recent research 
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on the effects of teaching and learning with technology. Using the statistical re- 
sults of experimental and quasi-experimental published research, they con- 
cluded that there is a modest, positive effect of teaching and learning with tech- 
nology on student outcomes. The authors noted a lack of quality, refereed 
quantitative studies in the area of educational technology and alluded to the se- 
rious problem of the lack of empirical evidence that certain programs or ap- 
proaches utilizing technology are effective. 

Although the results of many research studies and meta-analyses do show 
some positive results in the use of technology, policymakers want to see it deci- 
sively demonstrated that technology’s value measures up to the cost. They are 
demanding evidence that their investments in educational technology have been 
worthwhile. The U.S. Department of Education has been rallying educators 
and researchers to marshal evidence that shows that students benefit from a 
high-tech environment. A recent paper commissioned by the U.S. Department 
ofEducation titled >4 Retrospective on TwentyVears of Education TechnoiogyPoiicy 
(Culp, Honey, & Mandinach, 2003) examined twenty years of key policy re- 
ports. Providing an overview and analysis of 28 reports, the authors concluded 
that, “The call for research on the impact of educational technology on schools 
and teaching and learning activities is a final constant theme found over the 
past twenty years of reports” (p. 15). 

This mandate for research on the effects of technology on teaching and learn- 
ing is clearly confirmed in the National Educational Technology Plan (U.S. De- 
partment ofEducation, 2000), “There is a pressing need for a high-quality, 
long-term national agenda for collecting, analyzing, and disseminating informa- 
tion on the use and effectiveness of technology in education” (p. 44). Moreover, 
one of the five goals outlined in the national plan is to provide “research and 
evaluation activities that will improve the next generation of technology appli- 
cations for teaching and learning” (p. 4). 

The question is, however, “what research will most benefit educators and 
where should research efforts be directed?” The call for research in educational 
technology is clearly evident, but there is a need for a framework for that re- 
search. This paper reports on the results of a national study that was conducted 
to provide guidance for the development of an educational technology research 
agenda. 

The study, utilizing the Delphi technique, was undertaken to identify, catego- 
rize, and prioritize research needs that should be addressed in educational tech- 
nology over the next five years. A Delphi panel of 30 educational technology 
experts throughout the United States was formed to generate, discuss, and rate 
research priorities in educational technology. This e-research (Anderson & 
Kanuka, 2003) study featured a three-round Delphi process completed via the 
Internet. 

METHODOLOGY 

The study employed the Delphi technique to obtain a consensus from educa- 
tional technology experts about areas/issues that are most in need of research 
over the next five years. Initially developed by the RAND Corporation in the 
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early 1950s to predict future defense needs (Cope, 1981), the Delphi has now 
been implemented across discipline areas — and in education as early as 1971 — 
as a means of obtaining opinions from persons without physically bringing 
them together (Cyphert & Gant, 1971). 

Linstone andTuroff (1978) described the utility of the Delphi as a research 
technique particularly in the following situations: 

1 . The problem does not lend itself to precise analytical techniques but can 
benefit from subjective judgments on a collective basis. 

2. The individuals who need to interact cannot be brought together in a face- 
to-face exchange because of time or cost constraints. Further, a conven 
tional conference tends to be dominated by particularly strong personalities 
or to give rise to an undesirable bandwagon effect, (p. 275) 

A study examining the effectiveness of the Delphi in comparison to tradi- 
tional discussion groups determined that the Delphi was the more effective be- 
cause the “...anonymity and isolation of the participants facilitated a freedom 
from conformity pressures” (Van de Ven & Delbecq, 1974, p. 619). An addi- 
tional perceived benefit of using the Delphi is the belief that the writing process 
enables participants to thoroughly deliberate and reflect upon all aspects of the 
problem. The result is participants’ submission of precise, distinctive ideas. 

While there has been considerable variance in administering the Delphi pro- 
cess, prescribed methodology requires that two cycles of questionnaires and 
feedback reports be used (Van de Ven & Delbecq, 1974). For this study, former 
Preparing Tomorrow’s Teachers to Use Technology (PT^) Grant Directors 
throughout the United States were e-mailed an invitation to participate in a 
three-round Delphi process involving two cycles of online questionnaires and 
feedback reports. Procedural steps were as follows: 

Round 1: Participants were directed to the Delphi study Web site and asked 
to generate responses to the question, “What should the research priorities for 
educational technology over the next five years be?” Round 1 statements were 
arranged in categories according to research focus. Identified research state- 
ments and categories were then used to develop the Round 2 instrument. 

Round 2: Participants were asked to rate the research statements and catego- 
ries identified in Round 1 as to research need. In addition to rating each re- 
search statement, they ranked the major research categories in order of their 
perceived importance. Once returned, descriptive statistics for the group ratings 
were calculated: mean, median, and standard deviation. 

Round 3: The ratings of research statements and rankings of major research 
categories by the group in Round 2 were compiled. Participants in Round 3 
again ranked the major research categories as they did in Round 2, but this time 
descriptive information about how the group responded, as a whole, was pro- 
vided. Participating experts were asked to review each item, consider the group 
response and then re-rate the items, taking the information into account. 

The three-round Delphi process enabled the participants to generate their 
own opinions about necessary educational technology research areas, prioritize 
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research focus categories, and then to finalize their views based upon consider- 
ation of the entire group’s opinions. This process, engendering the dynamics of 
effective group interactions, enabled researchers to gain a consensus from a 
panel of expert participants in diverse geographical locations about educational 
technology research priorities for the twenty-first century. 

Delphi Panel Experts 

The educational technology experts invited to participate in this Delphi study 
were individuals who had served as grant directors for a Preparing Tomorrow’s 
Teachers to Use Technology (PT^) grant. The PT^ grant program (U.S. Depart- 
ment of Education, 2004) was created by Congress in response to the urgent 
call for teacher quality improvement and the need to reform teaching and learn- 
ing in education. Since 1999, PT^ has awarded more than 400 grants to innova- 
tive programs focusing on technology-infused learning in the K— 16 environ- 
ment, with formal and informal evaluation of grant interventions as integral 
elements. Although specific program components differed, each of the PT^ 
grant awardees worked to transform education so that technology became inte- 
grated throughout the teaching and learning process. PT^ grant awardees were 
required to provide a research basis for all grant activities, develop strong grant 
evaluation components, and use sound research methodology in examining the 
effects of technology interventions. Thus, PT^ grantees were viewed as the most 
appropriate choices for participating on the Delphi panel based on their proac- 
tive involvement in promoting technology-infused learning in the K— 16 envi- 
ronment and in examining the effects of those efforts. 

As the Delphi process is a time-consuming one, potential expert participants 
were contacted in advance to determine if they were willing to commit to all 
three rounds of the research study. Sixty-three grant directors from the 1999 
PT^ grant program were contacted by e-mail and asked to particpate or to rec- 
ommend someone from their grant (e.g., the grant evaluator) who was actively 
engaged in the grant technology research activities to take part in the study. 
Thirty-two educational technology experts initially agreed to participate, but 
only 30 were able to complete all three rounds of the Delphi process. A review 
of Delphi studies reveals that Delphi panels are usually comprised of 10 to 20 
members and that “few new ideas are generated within a homogeneous group 
once the size exceeds 30 well-chosen particpants” (Delbecq,Van de Ven, & 
Gustafson, 1975, p. 89). 

Table 1 provides the demographic characteristics of the 30 Delphi expert par- 
ticipants, who were primarily (90%) comprised of university professors and ad- 
ministrators. The 19 males (63%) and 1 1 females (37%) were experienced tech- 
nology users, with 26 participants (87%) reporting more than 10 years of 
technology experience. 

FINDINGS 

The results of the three-round Delphi study reflect the consensus of opinions 
from 30 professionals in educational technology, all individuals who had 
worked as grant directors or grant evaluators for PT^ grants. The Round 1 ques- 
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Table 1; Demographic Information of Educational Technology Experts 
Completing the Delphi Study 


Demographic Item 

N 

% 

Gender 

Male 

19 

63 

Female 

11 

37 

Education 

PhD 

19 

63 

EdD 

8 

27 

DPA 

1 

3 

Masters 

2 

7 

Current Position Held 

Professor 

16 

53 

Administrator 

9 

30 

Professor/Administrator 

5 

17 

Years of Technology Experience 

5-9 years 

4 

13 

10-15 years 

8 

27 

16-20 years 

13 

43 

Over 20 years 

5 

17 

Institutional Affiliation 

University 

27 

90 

State Department 

2 

7 

Business 

1 

3 

Geographical Location 

Eastern U.S. 

11 

40 

Central U.S. 

9 

27 

Western U.S. 

10 

33 


tionnaire asked Delphi panel members to respond to the question, “What 
should be the research priorities for educational technology over the next five 
years?” In Round 1, expert participants generated 167 responses as to the re- 
search needs in educational technology over the next five years. These responses 
were reviewed by the researchers and consolidated so as to eliminate duplica- 
tion. Common research theme areas with similar responses were compiled with 
as much of the respondent’s original wording as possible retained. In total, 84 
statements were compiled and then categorized under eight research priority 
theme areas: Assessment, External Influences, Current Issues, Learning, Mod- 
els/Strategies, Schools, Teachers, and Web-Based Environments. 

For Round 2, panel members were asked to rate the 84 statements on a 
Likert-type scale as to the degree of need ( 1 = No Need; 2 = Low Need; 3 = Me- 
dium Need; 4 = High Need; 5 = Very High Need) for research that each state- 
ment represents. Additionally, panel members were asked to prioritize the eight 
research areas and encouraged to comment on any of the research statements 
and/or areas. This was their first exposure to the research statements and areas 
generated by the panel, and they were informed that they would have another 
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opportunity to assign ratings in Round 3. This round allowed them to assign 
their initial perception of the need for the research knowing full well that they 
would finalize their ratings based on their perceptions and the panel’s ratings in 
the next round. In this way, a group consensus on research priorities could be 
reached. 

The Round 3 questionnaire featured the panel ratings for the 84 statements 
and eight research areas listing the mean, median, mode, and standard devia- 
tion. For this round, panel members were asked to review the research priori- 
ties, consider the group response and then re-rate the items, taking the informa- 
tion into account. The eight priority areas were also reviewed and participants 
re-ranked the areas from 1 to 8, with 1 being the top priority for research in 
educational technology. 

Table 2 presents the Delphi panel’s ranking of the research priority areas for 
educational technology over the next five years. Research in all aspects of learn- 
ing and the effects of technology on the learning process was assigned the top 
priority, with research focused on teacher use of technology and teacher training 
as the second priority. 


Table 2; Ranking of Research Priority Areas— Round 3 D elphi Responses 

Rank 

Category 

Mean 

Mode 

N 

% 

1 

Learning 

1.32 

1 

22 

73 

2 

Teachers 

2.68 

2 

11 

37 

3 

Models/ Strategies 

3.63 

3 

10 

33 

4 

Assessment 

3.89 

3 

10 

33 

5 

Current Issues 

5.21 

5 

11 

37 

6 

Schools 

5.57 

6 

13 

43 

7 

Web-Based Environments 

6.21 

7 

11 

37 

8 

External Influences 

7.47 

8 

27 

90 


The rest of this paper will discuss the research priority categories and the spe- 
cific research statements within each priority area. Table 3 lists the 12 research 
statements included in the learning category with the Round 3 descriptive sta- 
tistical summary. 

Learning 

Although the need to investigate the impact of technology on student achieve- 
ment has been a common focus of technology research, the Delphi panel members 
still considered it a high priority for the next five years. Panel members commented 
on the establishment of federal, state, and local standards and accountability issues 
as additional reasons why it is so important to examine student achievement. 

An examination of the mean ratings of the 12 research statements generated 
discloses a wide range of perceived need, with the panel members emphasizing 
broader, more comprehensive research on learning and student achievement 
(mean = 4.47) and less emphasis on investigations to determine the difference 
made by employing specific technologies (mean = 2.57). They advocated re- 
search examining how people learn using technology, unlike many past research 
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studies that have focused on specific technologies and their impact on learning 
a particular set of skills or content (Honey, Culp, & Carrigg, 2000). The 
Delphi panel recommended that research efforts be directed toward the role of 
technology in helping students to become better problem solvers and to accom- 
plish learning tasks. Coupled with this examination was the need to design ef- 
fective technology-enhanced learning environments to determine how technol- 
ogy contributes to student learning. 

Panel members cited the need to conduct research that examines the impact 
of technology in relation to learning principles, brain research, and multi-modal 
learning. They discussed the need to develop an understanding of how people 
learn using technology and the impact of technology on the various domains of 
learning (cognitive, affective, and psychomotor). The emphasis was on “learn- 
ing” and not on scores on achievement tests. 

Table 3; Learning Research Category— Round 3 Delphi Responses 


Research Need 

Mean 

SD 

Mdn 

Investigate the impact of technology on student 
achievement 

4.47 

.69 

5 

Develop and use learning principles to design 
technology-enhanced learning environments 

4.10 

.71 

4 

Examine the role of multi-modal learning to provide 
“just in time” and only “just as needed” solutions 
to training and informational needs 

3.85 

1.08 

4 

Link knowledge about teaching and learning 
with appropriate technologies 

3.95 

.60 

4 

Determine the efficacy of learning content using 
technology versus more traditional methods 

3.45 

1.09 

4 

Examine methods to learn technology contextually 

3.40 

.99 

3 

Determine if the level of technology use relates to 
the engagement of activities in the learning process 

3.30 

.57 

3 

Determine the impact of technology on various 
domains (cognitive, affective, psychomotor) 

3.25 

.71 

3 

Identify a taxonomy that best represents the issues 
and understandings of learner centeredness 
and deeper learning 

3.25 

.78 

3 

Examine methods to make students better at solving 
problems and accomplishing tasks using ICT 
(Information and Communication Technology) 

3.25 

.98 

3 

Examine how the brain learns and use brain 
science-based understanding of how to make effective 
use of ICT to help improve learning 

3.15 

.93 

3 

Investigate the cognitive paths of learners in interactive 
courses 

3.10 

.96 

3 

Determine which technologies are making a difference 

2.57 

1.3 

2 


Note Participantswere asked torateeach research statement on the basisof need usinga 5-polnt 

Likert-typescale (1 = NoNeed and 5 = l/ery H igh N eed). 
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Teachers 

Research designed to develop models for preparing inservice and preservice 
teachers to be more effective users of technology was seen as a high priority area. 
Most of the panel members were directly involved in teacher training and perceived 
a need for research-based models for teacher training and professional development 
activities. Table 4 lists the eight recommendations generated by the group. 

Table 4; Teachers Research Category - Round 3 Delphi Responses 


Research Need 

Mean 

SD 

Mdn 

Develop models for preparing inservice and preservice 
teachers to be more effective users of technology 

4.05 

.68 

4 

Examine approaches that apply technologies to 
individualize teacher professional development based 
on real problems and opportunities in the teacher’s 
classroom 

4.0 

.72 

4 

Examine technology barriers to assist inservice and 
preservice teachers when integrating educational 
technologies into the daily teaching and learning 
environment 

3.85 

.81 

4 

Examine the nature and extent of ICT (Information 
and Communication Technology) in preservice 
teacher education to produce teachers who effectively 
integrate technology (instruction and assessment) 
as soon as they get on the job 

3.6 

.82 

3.5 

Examine effective technology professional development 
practices in K— 12 schools 

3.45 

.83 

3 

Examine teaching styles in relation to the use of 
technology 

3.35 

.75 

3 

Determine barriers to using technology in the 
classroom (hardware availability, time, etc.) 

3.15 

.93 

3 

Determine the influence of handhelds (Palms) on 
critical thinking and teacher behavior 

2.95 

.88 

3 


Note Participantswere asked torateeach research statement on the basisof need usinga 5-point 

Likert-typescale (1 = NoNeed and 5 = Very H igh N eed). 


The Delphi panel experts not only expressed a need to develop models for 
preparing technology-proficient preservice teachers, but wanted to examine the 
nature and extent of ICT (Information and Communication Technology) in 
preservice teacher programs to produce teachers who effectively integrate tech- 
nology as soon as they begin their teaching duties. 

Acknowledging that there are barriers to integrating technology, the panel recom- 
mended research to examine those barriers in order to develop ways to assist 
inservice and preservice teachers. As one panel member wrote, “Teachers are the 
key to technology integration. If we want them to integrate technology, they need 
to help in overcoming the many challenges associated with using technology.” 
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M odel^Strategies 

The focus of the recommendations in this category was to examine ways to im- 
prove teaching/learning through the use of technology-based instructional models 
and strategies. The Delphi panel generated 15 research statements that were in- 
cluded in a category titled Models/Strategies. Although the members of the Delphi 
panel generated more research statements within this category than any other in 
the study, it should be noted that no mean rating for individual research statements 
was over 4.0, as was evidenced in some of the other research categories. The fact 
that the panel members were, on the whole, heavily involved in teacher training 
could account for the preponderance of suggested research activities. Table 5 lists 
the 1 5 statements and descriptive statistical summary for each statement. This re- 
search priority area featured recommendations from the educational technology ex- 
perts to examine the use of technology to develop instructional models and strate- 


TableS; Models/Strategies Research Category- Round 3 Delphi Responses 


Research Need 

Mean 

SD 

Mdn 

Examine and develop models for integrating technology 
into K— 12 instruction 

3.95 

.82 

4 

Examine the use of technology to support distributed 
learning and project-based learning 

3.85 

.87 

4 

Determine “best practices” scenarios for teaching 
with technology 

3.8 

1.00 

4 

Examine the use of technology enhanced instructional 
strategies to benefit specific groups of students 

3.6 

.59 

4 

Investigate ways to use technology to extend the 
learning experience beyond the traditional classroom 

3.6 

.88 

4 

Develop effective educational simulations 

3.4 

.94 

3.5 

Determine effective use of tool software to support 
the teaching and learning process 

3.3 

.73 

3 

Create interoperable technology applications 

3.2 

.95 

3 

Determine the impact of digital engagement practices 
(i.e., digital storytelling and ThinkQuest) on K-12 
student achievement 

3.3 

.80 

3 

Develop more effective software to promote student 
learning 

3.15 

.87 

3 

Determine areas in which Highly Interactive Computer- 
Assisted Learning is more effective than traditional 
teaching 

3.1 

1.02 

3 

Restructure “tutoring” software to learn from the student 

3.05 

1.05 

3 

Examine the role of “expert systems” to enhance learning 

2.95 

.82 

3 

Develop intelligent tutoring systems for appropriate 
higher ed courses and programs 

2.95 

1.05 

3 

Determine effective use of peripheral devices for 
supporting teaching/learning 

2.75 

.71 

3 


Note Participantswere asked torateeach research statement on the basisof need usinga 5-polnt 

Likert-typescale (1 = NoNeed and 5 = l/ery H igh N eed). 
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gies to support learning. Moreover, recommendations to determine the best use of 
technology to support specific learning models (distributed and project based 
learning) were cited by the experts. 

Although the investigation of specific technologies was given less priority, 
Delphi panel members did suggest research in determining the effective use of 
tool software and peripheral devices. Examining the effects of digital engage- 
ment practices (i.e., digital storytelling and Thinkquest), tutoring software/sys- 
tems, and expert systems were also suggested as research topics. 

Assessment 

The need to determine methods and criteria for evaluating the efiecriveness of technol- 
ogy enhanced instmaion, including assessment that is multifaceted, were priority topics 
included in the assessment category. Table 6 presents the 10 research statements gener- 
ated by the Delphi participants, with mean ratings from 2.95 to 4.35. 

Table 6; Assessment Research Category - Round 3 Delphi Responses 


Research Need 

Mean 

SD 

Mdn 

Determine methods and criteria for evaluating the 
effectiveness of technology-enhanced instruction 

4.35 

.74 

4.5 

Employ assessment that is multifaceted (not just 
focused on high-stakes testing) 

4.25 

.71 

4 

Determine the effectiveness of using electronic 
teaching portfolios to assess performance in meeting 
the various standards 

3.8 

1.0 

4 

Create an infrastructure for using technology to support 
teaching rigorous content and formatively assess 
student learning with respect to this content 

3.65 

.98 

4 

Examine the use of technology to enhance alternative 
forms of assessment 

3.6 

.75 

4 

Determine performance outcomes of student use of 
technology tools in academic areas 

3.3 

.73 

3 

Develop appropriate (fair, valid, reliable, cost-effective) 
assessment of student use of ICT as an aid to solving 
problems and accomplishing tasks in all subject areas 

3.2 

.89 

3 

Examine the role of non-intrusive, real-time assessment 
of student learning in optimizing time-on-task and 
teacher efficacy 

3.2 

.89 

.3 

Validate assessments for determining technological 
knowledge, skills, and dispositions 

2.95 

.68 

3 

Examine the use of PDAs for assessment 

2.95 

.82 

3 


Note. Participants were asked torateeach research statement on thebaa'sofneed usings 5-point 
Likert-typescalell =NoNee:land5 =VeryHi^ Need). 

Current Issues 


In this category, Delphi panel members recommended that research be con- 
centrated primarily on digital divide issues, including ways to eliminate or 
lessen the effects of the digital divide. Other issues included changes resulting 
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from the use of technology, including the effects on social interaction and col- 
laboration. The student use of computers outside of the school day was also 
suggested as an area in need of further research. The seven research statements 
are listed in Table 7 below. 

Table?; Current Issues Research Category— Round 3 Delphi Responses 


Research Need 

Mean 

SD 

Mdn 

Develop strategies to eliminate or lessen the effect 
of the digital divide 

3.95 

.82 

4 

Examine digital equity issues relating to low SES, 
gender and ethnicity 

3.65 

.93 

4 

Examine the effect of technology on social interaction 
and collaboration 

3.6 

.68 

3.5 

Determine factors influencing the digital divide 

3.4 

1.04 

3 

Examine human-computer interactions (information 
retrieval, immersive environments, usability, etc.) 

3.35 

1.03 

3 

Determine the ability of technology to engage students 
in important activity outside of the normal school hours 3.2 

.95 

3 

Determine the effects of educational technologies in 
a variety of “one-to-one” student/ computer 
environments (home and school) 

3.15 

.87 

3 


Note Partidpantswere asked torateeach research statement on thebasisofneed usings 5-point 

Likert-typescale (1 = NoNeed and 5 = Very H igh N eed). 

Schools 


The role of technology in lostering school improvement and ultimately student learn- 
ing were emphasized in the Schools research area (see Table 8). Of highest priority was 
the need to investigate the role of technology in the change process, with Delphi panel 
experts citing the need to examine ways in which technology can facilitate educational 
transformation, assist in assessing student growth, and help in making effeaive data- 
driven decisions. The Delphi panel advocated research to determine the crucial role of 
technology in facilitating educational transformation and school reform. 

Tables; Schools Research Category— Round 3 Delphi Responses 


Research Need 

Mean 

SD 

Mdn 

Investigate changes in classroom, teacher roles, and 
schools due to technology integration 

4.2 

.61 

4 

Investigate the effects of new models of schooling 
(using technology) to develop higher-order outcomes 
and critical attributes (i.e., creativity, problem solving, 
teamwork, etc.) 

4.0 

.74 

4 

Examine the role of technology in helping schools 
look at more than student performance in school 
improvement planning 

3.5 

.82 

3 

Examine the role of information technology in 
facilitating educational transformation 

3.5 

.82 

3 
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Tables con 't 


Research Need 

Mean 

SD 

Mdn 

Examine the role of technology in broadening the 
definition of Annual Yearly Progress and building 
school “report cards” that reflect student growth 

3.5 

.88 

3 

Determine the effect of administrative data-driven 
instructional decisions on student/school/classes 

3.5 

.94 

3 

Determine the best uses of computer mediated 
communication among schools to encourage and 
support innovation/research 

3.45 

.83 

3 

Examine the role of change theory and models to foster 
changes in schools when integrating technology 

3.4 

.75 

3 

Determine the use and effectiveness of technology 
in rural schools 

3.3 

.97 

3 

Determine the impact of administrative support on 
technology implementation 

3.25 

.85 

3 

Examine the role of funding in the support and use 
of technology 

2.95 

.82 

3 


Note. Participantswere asked torateeach research statement on the basis of need usings 5-point 


Likert-typescale (1 = NoNeed and 5 = Very H igh N eed). 

Web-Based Environments 

Table 9 lists the Delphi panel recommendations for research to be conducted 
within Web-based environments. Of top priority is the need to investigate the 
online collaborative leaning process in an effort to develop collaborative tools to 
support that process. Research to determine effective online instructional models 
and the effects of online instruction on student learning were advocated as well. 

Tables; Web-Based Environments Research Category— Round 3 Delphi 
Responses 


Research Need 

Mean 

SD 

Mdn 

Investigate the online collaborative learning process 
Examine the challenges faced in moving from teacher- 
centric transfer model of learning to the design of rich, 

3.95 

.68 

4 

Web-based learning-centered environments 

Develop collaborative tools to support collaborative 

3.85 

.74 

4 

processes 

Examine the role and structure of effective online 

3.8 

.89 

4 

mentoring 

Examine the integration of models of instruction into 

3.7 

.86 

4 

Web-based instructional strategies 

3.65 

.74 

3.5 

Develop models for online instruction 

Determine the effect of virtual classrooms on student 

3.55 

.68 

4 

learning 

Determine the impact of e-learning on educational 

3.55 

.99 

4 

programs 

3.35 

.58 

3 
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Table 9 con 't 

Research Need 


Mean SD 


Mdn 


Determine the effects of connecting at-risk students 
with advocates who care about and communicate 

with the student electronically and not face-to-face 3.2 .61 3 

Examine the role of video literacy to support the 
replacement of current text-based Internet 

communications 3.05 .82 3 

Note Participants were asked torateeach research statement on thebaasofneed usinga 5-point 
Likert-typescale (1 = NoNeed and 5 = l/ery H igh N eed). 

External Influences 

Although external influences were recognized as having an effect on the use of 
technology in the schools, there were few areas considered as priority for re- 
search. Table 10 lists the recommendations, which received mean ratings of 
1.65 to 3.4, the lowest ratings of all the research categories. 

Table 10; External Influences Research Category— Round 3 Delphi Re- 
sponses 


Research Need 

Mean 

SD 

Mdn 

Examine the impact of Federal and state policy on the 
use of technology for teaching and learning 

3.4 

1.09 

4 

Identify uses of the Internet to improve educational 
reform and [QA: Areword/s missing here?] 

3.2 

.89 

3 

Develop policies to facilitate compatibility and 
interoperability between state accountability 
systems practices 

2.75 

1.02 

3 

Determine the impact of top-down approaches, at 
the federal and state levels, that lead to appropriate 
use of ICT (Information and Communication 
Technology) in schools 

2.7 

.80 

3 

Define and determine support for technology literacy 

1.65 

.81 

2 


Note. Participants were asked torateeach research statement on the baa's of need ua'nga 5-point 
Likert-type scale (1 =NoNeedand5 =VeryH Need). 


H ighest Priority Research Activities 

An examination of the statements included within each of the eight research 
priority categories reveals six specific research activities considered as the great- 
est priority. These six research statements, as listed in Table 1 1, received a mean 
of over 4.0, placing them in a very high-need rating. 

SUMMARY 

A review of educational technology literature over the past three decades re- 
veals a proliferation of research articles and national reports detailing the effects 
of computer technology in the classroom. In the 1970s and ’80s the majority of 
research efforts examined the effects of particular technologies or software on 
student learning (typically evidenced by student scores on a particular subject- 
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Table 11; Specific Research Statements of H ighest Priority - Round 3 
Delphi Responses 


Research Need 

Mean 

SD 

Mdn 

Investigate the impact of technology on student 
achievement 

4.47 

.69 

5 

Determine methods and criteria for evaluating the 
effectiveness of technology enhanced instruction 

4.35 

.74 

4.5 

Employ assessment that is multifaceted 

4.25 

.71 

4 

Investigate changes in classroom, teacher roles, and 
schools due to technology integration 

4.2 

.61 

4 

Develop and use learning principles to design 
technology-enhanced learning environments 

4.10 

.71 

4 

Develop models for preparing inservice and preservice 
teachers to be more effective users of technology 

4.05 

.68 

4 


Note. Participants were asked torateeach research statement on the basis of need usings 5-point 

Likert-typescale (1 = NoNeed and 5 = Mery H igh N eed). 


related test), student attitudes, and attendance (Honey, Culp, & Carrigg, 2000; 
Roblyer, 1988). The ’90s brought about some changes: the focus began to shift: 
from research on specific kinds of technology and their effects on student learn- 
ing to a broader examination of the effects of technology on the analytical and 
creative abilities of students. Moreover, new graphic-rich interactive technolo- 
gies have helped shape the direction of research efforts, with researchers investi- 
gating all facets of the online learning environment and the opportunities pro- 
vided by the Internet for teaching and learning. 

For the last 20 years, government-funded policy reports have repeatedly iden- 
tified the need for research on the effect of educational technology on teaching, 
learning, and schools to substantiate increased technology funding. Recent re- 
ports (Culp et ah, 2003) advocate the need to “establish a definition of condi- 
tions for effective use of technology; create new measures of progress and indi- 
cators of effective use; and design new approaches to assessment and more 
sensitive evaluation tools” (p. 16). These research priorities were identified by 
the Delphi panel members as well. 

The TedCher^ Tools for the 21^ C entury report (National Center for Educa- 
tion Statistics, 2000) espoused nine research questions to guide educational 
technology research efforts. Among the research focus areas were the need 
to determine types of technologies available in schools, changes to enable 
increased use of technology, fiscal expenditures on technology at all levels, 
and the benefit of technology in terms of costs. Although the Delphi panel 
members did not address these more monetary issues, they did echo the 
report’s stated need for research in technology as it relates to learning, 
teacher training, assessment, and the effect of technology on schools and 
educational reform. 

Specifically, the members of the Delphi panel advocated a research agenda 
that includes an examination of the following areas: 
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1. Lesming — examine the relationship of technology and how people learn in 
eluding an investigation of the learning process, learner engagement, and 
contextual learning. 

2. Teachers — develop models for preparing inservice and preservice teachers to 
be more effective users of technology. 

3. M oddS^StrateyeS — develop technology-rich instructional models to sup 
port student learning in the classroom and in the online environment. 

4. Assessnent — develop appropriate methods and criteria for evaluating the 
effectiveness of technology-enhanced instruction, particularly for more 
complex learning tasks. 

5. Schools — investigate changes in the classroom, teacher roles, and schools 
due to technology integration and determine how technology might best 
facilitate educational reform. 

6. Social Issues — investigate factors influencing the digital divide and the ef 
fects of technology on social interaction and collaboration. 

The need for empirical research determining the effects of technology on teaching and 
learning has been well recognized. The members of the Delphi panel in this national 
smdy not only supported this premise, but advocated that there be a “rigorous documen- 
tation” of the link between technology use and learning. Given the complex namre of 
technology, they agreed that there should be a complex and blended approach to research 
that seeks understanding about a broad range of factors. Moreover, the panel members 
recommended that educational technology research methodology include the use of lon- 
gimdinal and mixed-methods research involving teachers and classrooms. The use of 
multiple methodologies and a triangulation of findings in educational research wiU pro- 
vide the “rigorous documentation” advocated by panel members. Furthermore, the re- 
sults of this smdy provide specific research priority areas and topics for those engaged in 
educational technology. 
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